Part 3: Advanced Remote Execution Tools

In the last section we trained a toy model using Federated Learning. We did this by calling .send() and .get() on our model, sending it to the location of training data, updating it, and then bringing it back. However, at the end of the example we realized that we needed to go a bit further to protect people privacy. Namely, we want to average the gradients before calling .get(). That way, we won't ever see anyone's exact gradient (thus better protecting their privacy!!!)

But, in order to do this, we need a few more pieces:

  • use a pointer to send a Tensor directly to another worker

And in addition, while we're here, we're going to learn about a few more advanced tensor operations as well which will help us both with this example and a few in the future!

Authors:


In [ ]:
import torch
import syft as sy
hook = sy.TorchHook(torch)

Section 3.1 - Pointers to Pointers

As you know, PointerTensor objects feel just like normal tensors. In fact, they are so much like tensors that we can even have pointers to the pointers. Check it out!


In [ ]:
bob = sy.VirtualWorker(hook, id='bob')
alice = sy.VirtualWorker(hook, id='alice')

In [ ]:
# this is a local tensor
x = torch.tensor([1,2,3,4])
x

In [ ]:
# this sends the local tensor to Bob
x_ptr = x.send(bob)

# this is now a pointer
x_ptr

In [ ]:
# now we can SEND THE POINTER to alice!!!
pointer_to_x_ptr = x_ptr.send(alice)

pointer_to_x_ptr

What happened?

So, in the previous example, we created a tensor called x and send it to Bob, creating a pointer on our local machine (x_ptr).

Then, we called x_ptr.send(alice) which sent the pointer to Alice.

Note, this did NOT move the data! Instead, it moved the pointer to the data!!


In [ ]:
# As you can see above, Bob still has the actual data (data is always stored in a LocalTensor type). 
bob._objects

In [ ]:
# Alice, on the other hand, has x_ptr!! (notice how it points at bob)
alice._objects

In [ ]:
# and we can use .get() to get x_ptr back from Alice

x_ptr = pointer_to_x_ptr.get()
x_ptr

In [ ]:
# and then we can use x_ptr to get x back from Bob!

x = x_ptr.get()
x

Arithmetic on Pointer -> Pointer -> Data Object

And just like with normal pointers, we can perform arbitrary PyTorch operations across these tensors


In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
p2p2x = torch.tensor([1,2,3,4,5]).send(bob).send(alice)

y = p2p2x + p2p2x

In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
y.get().get()

In [ ]:
bob._objects

In [ ]:
alice._objects

In [ ]:
p2p2x.get().get()

In [ ]:
bob._objects

In [ ]:
alice._objects

Section 3.2 - Pointer Chain Operations

So in the last section whenever we called a .send() or a .get() operation, it called that operation directly on the tensor on our local machine. However, if you have a chain of pointers, sometimes you want to call operations like .get() or .send() on the last pointer in the chain (such as sending data directly from one worker to another). To accomplish this, you want to use functions which are especially designed for this privacy preserving operation.

These operations are:

  • my_pointer2pointer.move(another_worker)

In [ ]:
# x is now a pointer to the data which lives on Bob's machine
x = torch.tensor([1,2,3,4,5]).send(bob)

In [ ]:
print('  bob:', bob._objects)
print('alice:',alice._objects)

In [ ]:
x = x.move(alice)

In [ ]:
print('  bob:', bob._objects)
print('alice:',alice._objects)

In [ ]:
x

Excellent! Now we're equiped with the tools to perform remote gradient averaging using a trusted aggregator!

Congratulations!!! - Time to Join the Community!

Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement toward privacy preserving, decentralized ownership of AI and the AI supply chain (data), you can do so in the following ways!

Star PySyft on GitHub

The easiest way to help our community is just by starring the Repos! This helps raise awareness of the cool tools we're building.

Join our Slack!

The best way to keep up to date on the latest advancements is to join our community! You can do so by filling out the form at http://slack.openmined.org

Join a Code Project!

The best way to contribute to our community is to become a code contributor! At any time you can go to PySyft GitHub Issues page and filter for "Projects". This will show you all the top level Tickets giving an overview of what projects you can join! If you don't want to join a project, but you would like to do a bit of coding, you can also look for more "one off" mini-projects by searching for GitHub issues marked "good first issue".

If you don't have time to contribute to our codebase, but would still like to lend support, you can also become a Backer on our Open Collective. All donations go toward our web hosting and other community expenses such as hackathons and meetups!

OpenMined's Open Collective Page